solving 3x3 complex matrix